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ABSTRACT 


The  purpose  of  this  thesis  was  to  design,  conduct,  analyze  and  report  the  results 
of  a  C3  experiment,  'ilie  T4  Simulation,  acting  as  a  surrogate  for  a  C3  system,  was  used 
to  generate  data  for  statistical  analysis.  The  objective  was  to  determine  which  factors, 
and  which  factor  levels,  effected  the  MOE.  As  a  result,  the  optimal  system  alignment 
wac  determined  which  would  result  in  maximum  values  for  the  MOE. 

The  factors  investigated  were  Area,  Communication,  and  Tactical  Delay,  and 
probability  of  winning  a  same  turn  conflict,  P(W).  The  levels  of  delay  varied  from  zero 
to  nine  moves  while  levels  for  P(W)  varied  from  zero  to  one  in  increments  of  one  tenth. 

Analysis  showed  that  only  Tactical  Delay  effected  the  MOE  and  that  only  two 
levels,  zero  and  one,  of  Tactical  Delay  significantly  changed  the  MOE.  Analysis  also 
showed  that  the  player  with  the  higher  value  of  P(W),  regardless  of  Tactical  Delay, 
achieves  a  positive  MOE.  Therefore,  the  optimal  system  alignment,  under  the 
constraints  of  the  experimental  design,  would  be  to  assign  P(W)  =  1.0  to  one  side,  while 
assigning  the  maximum  value  of  delay  to  the  other.  Thus,  this  game  configuration 


would  maximize  the  MOE. 
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I.  INTRODUCTION 


A.  BACKGROUND 

The  joint  Command,  Control  and  Communications  (C3)  curriculum  is  designed  to 
provide  a  comprehensive  operational  and  technical  understanding  of  the  field  of  C3 
systems  as  applied  to  military  operations.  Students  are  introduced  to  modeling,  systems 
architecture  and  engineering,  and  evaluation  of  C3  systems  along  with  a  basic  knowledge 
of  the  physical  principles  and  technologies  that  comprise  communications  systems, 
computers  and  sensors.  The  joint  C3  curriculum  includes  instruction  in  communications 
and  sensors,  information  science  and  tactical  analysis/operations  research  that  supports 
C3  systems  development  [Ref.  1:  p.  66]. 

B.  PURPOSE 

This  thesis  concentrates  in  the  area  of  C3  systems  evaluation.  The  Naval 
Postgraduate  School  (NPS)  offers  a  course  in  C3  systems  evaluation  primarily  intended 
for  students  in  the  C3  curriculum. 

The  course  is  designed  to  be  one  of  the  curriculum’s  capstone  courses.  C3 
experiments  related  to  the  C3  evaluation  process  are  conducted  during  the 
laboratory  portion  of  the  course.  [Ref.  2;p  2] 

The  C3  systems  evaluation  course  uses  experiments  involving  simulations  and 
wargames  to  help  evaluate  systems  effectiveness.  Typically,  we  evaluate,  compare  or 
test  hypothesis  about  C3  systems  or  modifications  to  them.  C3  systems  evaluation  allows 
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the  student  to  leant  through  experience  how  to  design,  conduct,  analyze  and  report  the 
results  of  C3  experiments.  This  thesis  will  utilize  a  simple  game  for  the  purpose  of 
planning  and  conducting  an  experiment  including  hypothesis  development,  design  of 
experiment,  conduct  of  trials,  collection  and  analysis  of  data  and  reporting  of  the  results. 

An  enhanced  version  of  the  Tic-Tac-Toe  (TTT)  game  called  Tactical  Tic-Tac-Toe 
(T4)  was  developed  by  Gary  Porter  to  fulfill  the  need  for  the  first  experiment  in  the  C3 
Systems  Evaluation  course.  It  is  used  because  it: 

1.  is  easy  to  learn, 

2.  is  new  to  all  subjects, 

3.  is  a  non-military  game,  and 

4.  generates  data  that  are  C3  related. 

T4  is  a  two-sided  game.  Players  are  assigned  missions  to  achieve  and  their  score 
is  based  on  how  well  they  achieve  them.  Players  can  play  individually  against  an 
individual  opponent  or  can  be  in  teams  of  two  playing  against  other  teams  of  two. 
Missions  are  either  to  defeat  or  tie  your  opponent  in  the  assigned  game  board  (mission) 
area. 

Mission  effectiveness  depends  on  the  degree  to  which  you  have  achieved  your 
assigned  mission.  A  victory  mission  requires  that  you  score  more  TTTs  in  the  assigned 
mission  areas  than  your  opponent.  While  a  survival  mission  requires  you  to  at  least  tie 
your  opponent  in  the  assigned  mission  area  in  order  to  achieve  your  mission.  Mission 
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success  is  based  on  the  total  number  of  TTTs  achieved  in  an  assigned  mission  area 
relative  to  your  opponent. 

When  large  amounts  of  experimental  data  are  required,  using  human  subjects  to 
play  T4  and  to  collect  the  data  is  time  consuming  and  error  prone.  To  alleviate  these 
problems,  Porter  wrote  the  T4  Simulation  which  simulates  human  play,  generates  a 
wealth  of  data  and  collects  the  data  for  analysis.  The  data  generated  enable  the  user  to 
statistically  study  how  different  inputs  into  the  T4  game,  the  surrogate  for  a  C3  system, 
effect  mission  outcome. 

1.  Research  Questions 

The  course  in  systems  evaluation  referred  to  earlier  is  a  C3  course 
appropriately  called  C3  Systems  Evaluation.  The  course  requires  students  to  participate 
in  and  lead  an  experiment  that  uses  either  wargames  or  simulations  to  generate  data  for 
analysis.  It  is  in  this  course  that  students  are  exposed  to  the  T4  simulation.  The  course 
is  not  long  enough  for  students  to  pursue  a  lengthy  experiment.  Thus,  many  of  the 
questions  a  student  may  want  to  attempt  to  answer  may  not  be  considered.  It  is  for  that 
reason  this  thesis  is  being  written. 

This  thesis  will  investigate  how  the  T4  simulation  responds  to  different  types 
of  intelligence  delays.  The  delays  will  be  investigated  at  several  levels.  The  effect  of 
changing  the  probability  of  winning  a  same  turn  conflict,  P(W),  will  also  be  investigated. 
Finally,  this  thesis  will  investigate  the  effect  delay  and  P(W)  combined  have  on  mission 
outcome.  The  following  research  questions  will  be  answered  using  data  generated  as  a 
result  of  multiple  T4  replications: 
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1.  As  delay  levels  increase,  mission  margins  of  victory,  MOVs,  decline.  Is  there 
a  level  of  delay  above  which  further  increases  have  no  further  effect  on  mission 
MOV? 

2a.  Does  changing  the  probability  of  winning  a  same  turn  conflict,  P(W),  effect 
MOV? 

2b.  As  the  level  of  P(W)  decreases,  is  a  level  reached  beyond  which  P(W)  has  no 
effect  on  MOV? 

3.  Does  changing  the  probability  of  winning  a  same  turn  conflict,  P(W),  effect 

mission  outcome  regardless  of  Time  Delay? 

4.  Does  an  interaction  exist  between  Time  Delay  and  P(W)? 

These  research  questions  require  some  discussion  so  the  reader  is  clear  on 
exactly  what  is  to  be  accomplished  in  this  thesis.  Mission  outcome  refers  to  the  number 
of  TTTs  that  a  team  scores.  Relative  to  the  opponents  TTTs,  this  difference  in  scoring 
is  referred  to  as  the  MOV.  As  intelligence  delays  are  increased  for  a  particular  side,  it 
is  expected  that  the  number  of  TTTs  scored  by  that  side  will  decrease.  However,  it  is 
possible  that  beyond  a  certain  level  of  delay,  the  number  of  TTTs  will  not  continue  to 
decrease  but  will  level  off  at  some  value.  Delays  are  discussed  in  greater  detail  below. 

Conflict  resolution  is  also  described  in  detail  below.  So  for  now  it  will  just 
be  said  that  during  T4  games,  because  of  the  simultaneous  move  rule,  it  is  possible  for 
two  opposing  players  to  attempt  to  enter  the  same  game  cell  on  the  same  move.  Since 
only  one  player  is  allowed  to  occupy  a  cell,  a  conflict  results.  In  the  past,  these  conflicts 
were  resolved  randomly  with  both  sides  assigned  the  same  probability  of  winning,  P(W) 
=  O.S.  The  game  now  allows  the  probabilities  to  be  weighted  favoring  a  particular 
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player.  The  thesis  will  evaluate  the  mission  outcomes  when  one  side  is  assigned  a 
greater  P(W)  than  the  other  side. 

The  MOV  for  the  different  P(W)-Time  Delay  combinations  may  indicate  that 
P(W)  for  example,  has  no  effect  on  MOV  at  some  levels  of  Time  Delay  but  has  a 
significant  effect  at  others.  These  P(W)-Time  Delay  combinations  that  significantly 
effect  MOV  are  called  interactions  and  the  data  will  be  analyzed  to  see  if  any  exist. 

The  reader  may  not  be  able  to  adequately  understand  the  posed  questions  until 
a  discussion  of  T4  is  presented.  A  thorough  presentation  of  T4  is  provided  below.  It 
begins  by  describing  the  simplest  of  the  many  T4  variants,  the  T4  Baseline  game,  and 
finishes  with  an  explanation  of  the  T4  Simulation.  After  the  reader  understands  the  T4 
game  better,  it  is  suggested  that  the  above  Research  Questions  subsection  be  reread. 

2.  Approach 

T4  simulation  was  used  to  generate  the  data  that  was  analyzed  to  answer  the 
above  research  questions.  The  first  question  was  addressed  in  a  previous  thesis  by 
Jeffrey  S.  Richardson.  Richardson  hypothesized  that  as  time  delay  levels  increase, 
mission  outcome  would  decrease  up  to  a  point  beyond  which  further  increases  would 
have  no  effect  on  mission  outcome.  While  Richardson’s  data  showed  a  significant 
relationship  between  Tactical  Delay  levels  of  zero  and  one,  it  failed  to  show  that  further 
increases,  at  some  point,  would  have  no  effect  on  mission  outcome.  In  fact,  his  data 
showed  ''ery  little  relationship  between  an  increase  in  time  delay  and  mission  outcome. 
It  is  believed  though,  that  there  is  a  delay  value  above  which  mission  outcomes  no  longer 
decline,  but  the  number  of  replications  of  the  T4  simulation  used  in  Richardson’s  thesis 
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(30  replications  per  trial  configuration)  was  insufficient  to  show  this.  Richardson’s 
experimental  design  will  be  used  to  answer  question  one  using  120  replications  instead 
of  the  30  that  he  originally  used.  If  levels  of  delay  are  found  beyond  which  mission 
accomplishment  ceases  to  decline,  then  those  greater  delays  will  be  omitted  from  this 
thesis  in  order  to  more  fully  concentrate  on  the  remaining  research  questions. 

The  remaining  research  questions  required  a  change  in  the  T4  software  to 
allow  the  user  to  input  different  levels  of  probability  for  same  turn  conflict  resolution. 
Before  the  software  change,  when  same  turn  conflict  was  resolved  randomly,  each  side 
had  the  same  probability  of  winning  the  conflict.  To  answer  the  second  research 
question,  this  thesis  investigated  what  would  happen  if  one  side  had  a  higher  probability 
of  winning  a  same  turn  conflict.  The  data  generated  to  answer  Question  2  was  used  to 
answer  the  remaining  research  questions. 

C.  TACTICAL  TIC-TAC-TOE  (T4) 

1.  T4  Baseline 

Several  variants  of  T4  exist  but  it  is  not  necessary  to  discuss  all  of  them  in 
order  to  understand  the  game.  In  the  next  few  sections,  variants  that  will  enable  the 
reader  to  understand  the  version  of  the  game  used  with  the  thesis  will  be  discussed.  The 
T4  Baseline  game  is  the  most  fundamental  of  all  the  versions  and  will  provide  the  reader 
with  a  good  foundation  to  understand  the  T4  game  variant  used  in  this  thesis,  T4 
Simulation.  The  T4  Baseline  game  is  very  similar  to  the  well  known  game  of  Tic-Tac- 
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Toe  (TTT)  and  it  uses  the  same  three  cell  by  three  cell  game  board.  The  differences  will 
be  explained  below. 

a.  Simultaneous  Moves 

Both  players  announce  their  moves  simultaneously  and  plot  them  on  the 

game  board. 

b.  Conflict  Resolution 

Because  moves  are  made  simultaneously,  there  is  a  good  possibility  that 
both  players  will  chose  the  same  cell.  When  this  occurs,  it  is  called  a  same  turn  conflict 
and  the  winner  is  chosen  randomly,  flipping  a  fair  coin.  The  winner  remains  in  the  cell 
while  the  loser  is  not  allowed  an  alternate  move  and  simply  loses  that  turn.  A  variant 
of  this  game  alternates  same  turn  conflict  resolution  between  players  after  the  first 
conflict  is  decided  randomly. 

c.  Scoring 

Unlike  the  game  of  TTT,  T4  does  not  end  when  the  first  TTT  occurs. 
Instead,  T4  is  played  until  all  nine  cells  are  filled.  When  the  game  board  is  filled,  the 
player  with  the  most  TTTs  wins  and  is  awarded  one  point.  Recall  there  are  eight 
possible  ways  to  make  a  TTT  (three  horizontal,  three  vertical,  and  two  diagonal).  It  is 
possible  for  a  player  to  get  all  eight  TTTs  if  every  move  results  in  a  conflict  and  the 
same  player  wins  every  conflict. 
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2.  T4  With  Intelligence  Delay  Factor 

This  version  is  the  same  as  the  baseline  game  except  for  the  addition  of  an 
enemy  information  delay  factor.  This  factor,  delays  knowledge  of  opponents  moves. 
The  level  of  delay  can  be  varied  from  zero  to  nine  moves.  The  delay  level  is  assigned 
before  the  game  starts  and  can  be  set  at  the  same  or  different  levels  for  both  players. 
When  intelligence  delay  is  introduced  into  the  game,  another  type  of  conflict  occurs  and 
is  termed  a  different  turn  conflict. 

A  player  that  is  assigned  a  one  move  intelligence  delay,  for  example,  will  r.ot 
know  which  cell  the  opponent  moved  into  until  the  following  move.  Therefore,  it  is 
possible  the  player  assigned  a  delay  may  choose  a  cell  that  is  already  occupied.  When 
this  occurs,  the  conflict  is  always  won  by  the  player  first  occupying  the  cell.  Like  the 
baseline  game,  the  loser  is  not  allowed  an  alternate  move  and  loses  that  turn. 

3.  Double  Game  Board  T4 

This  version  uses  two  TIT  boards  to  form  one  larger  board  three  cells  high 
and  six  cells  wide.  Although  it  is  considered  only  one  board,  the  sides  are  designated 

as  left  and  right  game  boards.  In  this  game  a  turn  consists  of  each  player  making  a  move 

« 

on  both  the  left  and  right  game  boards.  These  moves,  like  the  baseline  game  are  made 
in  secret  and  announced  simultaneously.  Likewise,  same  and  different  turn  conflicts  are 
resolved  as  in  the  baseline  game. 
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a.  Crossover  Scores 


Unique  to  the  double  game  board  version  are  TTTs  that  cross  from  one 
game  board  side  to  the  other.  These  are  called  crossover  TTTs  and  there  are  ten  possible 
ways  for  these  to  occur.  Realize  they  can  only  occur  horizontally  or  diagonally  but  not 
vertically. 

b.  Intelligence  Delays 

Like  the  T4  with  Intelligence  Delay  game,  the  double  game  board  version 
permits  intelligence  delays  of  zero  through  nine.  Different  delay  levels  are  player 
independent.  Different  delay  levels  are  also  game  board-side  independent  in  that 
different  delay  levels  may  be  assigned  to  different  sides  of  the  game  board.  All  four 
players  can  be  assigned  different  values  of  delay.  For  example,  the  X  player  on  the  left 
game  board  may  have  no  intelligence  delay  while  the  right  game  board  X  player  has  an 
intelligence  delay  of  several  moves. 

c.  Mission  Assignments 

The  assignment  of  missions  in  this  version  allows  players  to  achieve  an 
objective  within  a  specific  area  of  the  game  board.  The  objective  assignments  are  either 
victory  or  survival.  Victory  is  achieved  by  scoring  more  TTTs  than  an  opponent  in  an 
assigned  mission  area  while  survival  is  achieved  by  either  scoring  a  victory  or  tieing  an 
opponent  in  an  assigned  mission  area. 
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The  specific  areas  of  the  game  board  that  a  player  may  be  assigned  an 
objective  are  left,  right,  crossover  and  overall.  The  following  is  a  complete  list  of  the 
mission  areas  a  player  may  be  assigned  and  a  brief  description  of  how  each  is  achieved. 

1.  VL  =  Victory  left.  By  scoring  the  most  TTTs  on  the  left  side. 

2.  VR  =  Victory  right.  By  scoring  the  most  TTTs  on  the  right  side. 

3.  VC  =  Victory  crossover.  By  scoring  the  most  TTTs  in  the  crossover  area. 

4.  VO  =  Victory  overall.  By  scoring  the  most  total  TTTs. 

5.  SL  =  Survival  left.  By  not  losing  the  left  side. 

6.  SR  =  Survival  right.  By  not  losing  the  right  side. 

7.  SC  =  Survival  crossover.  By  not  losing  the  crossover  area. 

8.  SO  =  Survival  overall.  By  not  losing  overall. 

Each  player  can  be  assigned  up  to  four  missions.  Like  intelligence 
delays,  different  players  can  be  assigned  different  missions.  It  should  be  evident  at  this 
time  that  there  are  numerous  game  configurations  to  choose  from  in  Double  Game  Board 
T4. 

4.  Team  T4  Games 

Team  T4  is  a  natural  progression  of  the  double  game  board  version  and 
introduces  additional  players,  a  controller  and  three  types  of  intelligence  delay. 
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a.  Players 

Team  T4  is  made  up  of  two  teams  each  with  two  players.  The  X  team 
consists  of  an  X  Left  (XL)  player  and  an  X  Right  (XR)  player.  Similarly,  the  O  team 
is  made  up  of  an  O  Left  (OL)  player  and  an  O  Right  (OR)  player.  Mission  assignments 
are  always  the  same  for  both  players  on  the  same  team  but  opposing  teams  may  be 
assigned  different  missions.  This  game  is  played  like  the  other  versions  but  tends  to  get 
complicated  because  of  the  increased  number  of  delay  factors  which  will  be  discussed 
below.  To  deal  with  the  increased  complexity  of  the  game,  a  controller  is  assigned  to 
monitor  its  progress. 

b.  Controllers 

A  controller  administers  the  game,  ensuring  conflicts  are  resolved  fairly 
and  player’s  game  boards  are  adjusted  corra:tly  taking  into  account  the  various  delay 
factors.  Like  the  other  T4  versions  players  choose  their  moves  in  secrecy  but  instead  of 
announcing  them  to  their  opponent,  they  give  them  to  the  controller.  The  controller  then 
updates  each  individual  game  board  and  returns  it  to  the  player.  The  controller  also 
maintains  a  master  game  board  that  represents  ground  truth,  the  actual  status  of  the  game 
without  any  intelligence  delays.  The  intelligence  delays  that  have  been  introduced  into 
this  version  will  now  be  presented. 

c.  Delay  Factors 

Intelligence  delay  has  been  expanded  into  three  categories  and  they  are 
described  below. 
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1. 


Tactical  delay.  This  delay  refers  to  knowledge  of  own  enemy  move  information 
(the  opponent  on  the  same  side  of  the  double  game  board). 

2.  Area  delay.  This  delay  refers  to  partner’s  enemy  move  information. 

3.  Communication  delay.  This  delay  refers  to  partner’s  move  information. 

These  delays  put  constraints  on  the  timeliness  of  receiving  move 
information,  by  any  one  player,  from  the  other  three  players.  Each  of  the  four  players 
can  be  assigned  different  values  for  each  of  the  three  different  delay  types.  These 
values,  like  the  original  intelligence  delay,  can  vary  from  zero  to  nine  moves. 

d.  Summary 

At  this  point,  the  reader  should  feel  reasonably  comfortable  that  they 
have  an  understanding  of  the  Team  T4  game.  To  reinforce  what  has  been  discussed  thus 
far,  a  quick  review  of  the  different  factors  that  can  be  assigned  to  Team  T4  games  and 
the  different  levels  some  of  these  factors  can  take  are  summarized. 

Two  players  each  are  assigned  to  the  X  and  the  O  team.  The  teams  are 
divided  into  left  and  right  side  players.  Each  team  is  assigned  up  to  four  mission  areas 
with  an  objective  of  either  victory  or  survival  in  each  area.  Recall  that  both  sides  on  the 
same  team  are  given  the  same  mission  assignment  while  the  opposing  team  may  have  an 
entirely  different  assignment.  Recall  also  that  three  types  of  delays  can  be  assigned  to 
each  player,  each  varying  from  zero  to  nine  moves.  These  delays  can  be  different  for 
each  of  the  four  players. 
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Experience  has  shown  that  while  game  play  is  fairly  simple,  the 
administration  of  me  game  is  difficult,  time  consuming  and  susceptible  to  errors. 

5.  Auto  T4 

As  might  be  expected,  a  computer  assisted  automation  of  the  game  was 

developed  to  assist  the  controller.  This  program  is  called  Auto  T4. 

This  program  allows  pretrial  configuration  of  the  T4  games,  computer  assisted 
administration  of  game  play  and  automatic  scoring  and  collection  of  data.  It  is 
hoped  that  use  of  this  aide  by  lead  groups  in  1991  T4  experiments  will  permit  an 
increase  in  sample  size  and  a  reduction  in  data  errors.  [Ref.  3;p.  1] 

It  is  important  to  realize  that  other  than  what  is  described  directly  above, 
there  is  no  difference  between  Auto  and  Team  T4. 

6.  T4  Simulation 

A  natural  progression  from  the  Auto  T4  game  is  the  T4  Simulation  program. 
It  simulates  four  players  participating  in  Team  T4  games,  complete  with  mission 
assignments  and  intelligence  delay  constraints.  Since  Auto  T4  already  contained  the 
software  to  score  the  game,  to  collect  all  the  data  generated  by  the  simulation  and  to 
record  it  in  a  data  file,  the  T4  Simulation  program  was  produced  by  the  addition  of  two 
major  changes. 

The  first  addition  was  a  method  to  control  the  simulated  play  of  each  player. 
The  problem  with  simulating  human  play  was  creating  the  software  to  do  it.  To 
accomplish  this,  each  simulated  player  is  controlled  by  a  set  of  three  game  plan  matrices. 

The  second  addition  allowed  multiple  trial  runs  to  be  accomplished  without 
user  intervention  between  trials  and  between  configurations.  A  game  file,  that  contains 


13 


all  configuration  information,  allows  the  user  to  specify  multiple  trial  configurations  and 
the  number  of  replications  to  be  run  for  each  configuration. 


The  original  purpose  of  the  T4  simulation  was  to  act  as  a  surrogate  for  human 

play  in  a  pilot  experiment  to  discover  a  set  of  T4  variables  of  interest  for  later  use  in 

trials  with  human  subjects.  It  was  thought  that  this  would  be  beneficial  in  several  ways. 

Using  simulated  trials  is  extremely  cost  effective  in  terms  of  time  required. 

Simulation  permits  the  use  of  a  much  larger  sample  size  resulting  in  a  higher 
degree  of  resolution  of  results  inferred  from  data.  Use  of  simulated  players  thus 
adds  greatly  to  the  degree  of  control  and  independence  of  the  trial  outcomes. 
Humans  make  mistakes,  learn  from  experience,  play  differently  with  different 
partners  and  other  sometimes  unknown  factors  that  serve  to  add  noise  to  the  data 
in  which  you  are  trying  to  find  a  signal.  [Ref.  4:p.  1] 

This  chapter  introduced  the  T4  game  to  the  reader.  The  purpose  of  the  thesis 

was  discussed  including  the  research  questions  and  the  approach  taken  to  answer  them. 

The  following  chapter  will  describe  the  experimental  design  that  was  used  to  generate  the 

data. 
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n.  EXPERIMENTAL  DESIGN 


A.  SETUP 

In  presenting  the  setup  of  an  experimental  design,  the  physical  setup,  the  test 
subjects  and  any  special  equipment  should  be  described.  This  is  fairly  simple  when  a 
simulation  is  used  to  generate  the  data.  When  using  a  simulation,  all  that  is  required  is 
a  computer  and  the  necessary  software.  While  using  human  subjects  in  the  experiment, 
other  issues  such  as  assignment  of  teams  and  controllers,  scheduling  of  when  and  where 
game  play  will  be  conducted,  keeping  score  and  data  collection  must  be  addressed.  For 
completeness  though,  the  hardware  and  software  used  to  run  the  simulation  and  to  collect 
and  analyze  the  generated  data  will  be  described. 

1.  Physical 

Gary  Porter’s  T4  Simulation  was  run  on  a  Macintosh  microcomputer  in 
HyperCard  2.0.  T4  is  written  in  HyperCard.  It  contains  the  code  which  simulates  game 
play  and  then  sends  the  generated  data  to  a  data  file.  The  data  is  copied  to  disc  in  text 
format  and  then  imported  into  Minitab  or  Microsoft  Excel,  the  two  software  packages 
used  to  analyze  and  graph  the  data.  Minitab  was  used  as  a  statistical  tool  to  do  the 
majority  of  the  analysis  while  Microsoft  Excel  was  used  to  manipulate  the  data  and 
produce  the  graphs  shown  in  chapter  three. 
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2.  T4  Simulation 


Before  discussing  the  simulation  setup,  it  is  important  to  realize  that  this 
thesis  does  not  fully  utilize  the  full  potential  of  the  T4  Simulation  in  generating  different 
configurations.  As  the  simulation  setup  is  described,  it  will  be  pointed  out  where  the  full 
range  of  possible  parameters  was  not  used. 

The  setup  of  the  T4  Simulation  requires  the  assignment  of  the  following 
parameters.  They  need  not  be  assigned  in  the  order  listed  below,  but  they  all  must  be 
assigned  before  the  simulation  can  run. 

1 .  player  intelligence  delay  levels  (Area,  Communication  and  Tactical)  from  zero 
to  nine  turns, 

2.  three  player  strategy  matrices  which  control  simulated  player  game  play, 

3.  team  missions  (up  to  four  survival  or  victory  mission  areas), 

4.  the  number  of  replications  per  configuration,  and 

5.  methods  for  resolving  same  turn  conflicts. 

The  first  research  question  attempts  to  answer  a  question  asked  by  Richardson 
in  his  thesis.  Therefore,  his  T4  Simulation  setup,  except  for  increased  sample  size,  will 
be  used  to  generate  data  to  answer  Question  1  and  is  presented  below.  To  answer 
research  questions  two  through  four,  some  changes  to  Richardson’s  setup  were  required. 
Changes  to  his  setup  are  clearly  identified  as  they  are  discussed. 
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a.  Intelligence  Delays 

Team  X  was  chosen  as  the  control  team,  the  team  with  zero  information 
delay  throughout  the  experiment.  The  O  team  then,  received  information  delays  during 
the  experiment  and  as  such,  was  considered  the  treatment  team.  During  the  experiment, 
both  simulated  O  team  players,  left  and  right  side,  always  received  the  same  level  of 
information  delay.  Each  of  the  three  types  of  delay  were  investigated  separately.  For 
example,  when  Area  Delay  was  increased  from  zero  to  nine,  the  delay  levels  for 
Communication  and  Tactical  Delay  were  held  constant  at  zero.  All  nine  levels  of  delay 
were  investigated. 

When  data  was  generated  to  answer  questions  two  through  four,  only 
Tactical  Delay  was  considered,  the  other  two  types  of  delay  (Area  and  Communications) 
were  not.  For  Tactical  Delay,  only  levels  zero  through  three  were  investigated.  The 
reasons  for  this  will  be  discussed  in  the  next  chapter.  Therefore,  the  full  range  of 
available  parameters  was  not  utilized  while  generating  data  for  these  questions. 

b.  Simulated  Player  Strategies 

After  determining  the  control  and  treatment  team,  strategies  were 
assigned  to  control  the  simulated  play  of  each  player.  Discussed  briefly  in  the  first 
chapter,  strategy  is  controlled  by  a  set  of  three  game  plan  matrices.  These  game  plan 
matrices  include: 

1.  The  Cell  Game  Plan  matrix  which  determines  where  a  player  will  move  at  the 
start  of  a  game.  It  also  assists  in  resolving  same  turn  conflicts. 

2.  The  Regular  Game  Plan  matrix  which  controls  a  players  moves  on  the  left  and 
right  game  boards. 
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3.  The  Crossover  Game  Plan  matrix  which  controls  a  players  move  in  the 
crossover  area  of  the  game  board. 


The  game  plan  matrices  are  assigned  point  values  by  the  user  and  the 
simulation  uses  these  values  to  determine  how  the  simulated  players  move.  The  way  the 
players  move  represents  theii  strategy.  For  an  in  depth  discussion  on  how  game  plan 
matrices  control  simulated  players,  see  Ref.  4. 

Representative  categories  of  available  strategies,  based  on  the  values 
assigned  to  the  game  plan  matrices,  that  can  be  assigned  to  each  individual  player  are 
listed  below: 

1.  Overall  Offense:  Ensures  the  simulated  player  will  attempt  to  maximize  own 
TTTs  while  making  no  attempt  to  block  opponents  TTTs. 

2.  Balanced  Offense:  Ensures  the  simulated  player  will  primarily  attempt  to  score 
own  TTTs,  but  will  also  try  to  block  opponent’s.  If  there  is  simultaneously  a 
TTT  possible  for  the  player  and  the  opponent,  a  tie,  the  player  will  make  an 
offensive  TTT  instead  of  blocking  the  opponent’s  TTT. 

3.  Balanced:  This  is  the  same  as  Balanced  Offense  except  in  the  case  of  a  tie. 
When  a  tie  occurs,  the  choice  of  either  making  a  TTT  or  blocking  one  is 
random. 

4.  Balanced  Defense:  Ensures  the  simulated  player  will  primarily  attempt  to  block 
opponent’s  TTTs,  but  will  also  try  to  score  their  own.  If  a  tie  occurs,  the 
player  will  block  the  opponent’s  ITT  rather  than  make  one. 

5.  Total  Defense:  Ensures  the  simulated  player  will  attempt  to  block  opponent’s 
TTTs  while  making  no  effort  to  score  their  own  TTTs. 

6.  Random:  This  strategy  provides  the  player  with  no  strategy,  all  moves  are 
random. 
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The  strategies  that  were  assigned  for  this  experiment  were  offensive 
crossover,  OC,  for  both  left  side  players  and  defensive  crossover,  DC,  for  both  right  side 
players.  OC,  a  variant  of  the  Balanced  Offense  strategy,  encourages  the  left  side  players 
to  score  TTTs  in  both  the  left  and  crossover  areas  of  the  double  game  board.  DC,  a 
variant  of  the  Balanced  Defense  strategy,  encourages  the  right  side  players  to  block  their 
opponent’s  TTTs  at  the  expense  of  scoring  their  own  in  the  right  and  crossover  areas. 
These  simulated  player  strategies  are  realized  as  a  result  of  assigning  different  values  to 
the  game  plan  matrices.  It  should  be  noted  that  each  player  can  be  assigned  one  strategy. 

c.  Mission  Assignments 

The  mission  assignments  were  VL  and  VC  for  both  teams.  Because  both 
teams  were  assigned  identical  missions  the  possibility  of  same  turn  conflicts  was 
maximized.  It  was  expected  that  this  setup  would  be  best  for  determining  what  happens 
when  changing  the  P(W)  is  investigated.  Recall  the  full  range  of  possible  mission 
assignments  includes  eight  different  missions,  see  p.9.  The  simulation  allows  each  team 
to  be  assigned  up  to  four  missions.  Once  again  the  full  range  of  available  parameters 
was  not  utilized. 

d.  Replications 

The  number  of  replications  per  configuration  was  increased  from  30  to 
120  to  generate  data  to  answer  all  of  the  research  questions.  It  was  believed  that  a 
sample  size  this  large  would  result  in  a  higher  degree  of  resolution  of  the  results  inferred 
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from  the  data.  The  only  limit  to  the  number  of  replications  is  the  number  of  rows 
available  on  the  spreadsheet  used  to  analyze  the  data 

e.  Same  Turn  Conflict  Resolution 

Finally,  same  turn  conflict  resolution  was  random  for  all  conflicts,  initial 
and  subsequent.  To  answer  the  research  questions  two  through  four,  the  O  team  was 
assigned  a  different  P(W)  for  each  game  configuration  ranging  from  1.0  to  0.0  in  steps 
of  0. 1.  For  example,  the  O  team  for  the  first  120  replications  was  assigned  P(W)  =  1.0. 
The  X  team  then,  was  automatically  assigned  the  complementary  value  of  P(W)  =  0.0. 
This  probability  applied  to  both  initial  and  subsequent  same  turn  conflicts.  The  T4 
Simulation  program  user  can  assign  a  P(W)  from  1.0  to  0.0  for  the  first  conflict  that 
occurs.  All  subsequent  conflicts  can  be  resolved  either  randomly  or  alternately. 
Different  P(W)s  may  be  assigned  for  initial  and  subsequent  same  turn  conflicts. 

The  purpose  for  pointing  out  the  full  range  of  T4  configuration  options 
is  not  to  critique  the  thesis  but  to  point  out  that  many  more  configurations  are  available 
for  additional  study. 

B.  PROCEDURE 

After  the  generated  data  was  transferred  to  disc  it  was  reduced  and  manipulated 
in  Microsoft  Excel  and  put  into  columns  for  analysis.  Both  the  statistical  package  in 
Excel  and  Minitab  require  data  to  be  in  columns  for  subsequent  analysis.  The  raw  data 
extracted  from  the  original  spreadsheet  consisted  of  eight  columns.  Four  each  for  the  X 
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and  O  teams.  Each  four  column  group  was  labeled  Scoring  by  ms  and  the  individual 
columns  were  labeled  as  follows; 

1.  TL  which  represents  the  total  TTTs  achieved  on  the  left  side  game  board. 

2.  TC  which  represents  the  total  TTTs  achieved  on  the  crossover  area  of  the  game 
board. 

3.  TR  which  represents  the  total  TTTs  achieved  on  the  right  side  game  board. 

4.  TO  which  represents  the  total  TTTs  achieved  on  the  entire  game  board  is  the 
sum  of  the  first  three  columns. 

Since  the  assigned  missions  did  not  include  VR,  the  TR  values  were  eliminated  and 
TO  was  recalculated  using  only  the  TL  and  TC  columns.  The  margin  of  victory,  MOV, 
was  then  established  using  the  following  equation:  MOV  =  TO,  -  TOo.  In  other  words, 
the  MOV  is  the  difference  between  the  total  X  team  TTTs  and  the  total  O  team  TTTs 
in  the  assigned  mission  areas.  It  is  the  MOV  columns  that  are  statistically  analyzed  in 
this  thesis  and  no  further  data  manipulation  was  required. 

The  following  chapter  discusses  the  statistical  methods  used  to  analyze  the  MOV 
and  presents  the  results  of  the  analysis. 
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m.  ANALYSIS 


A.  ANALYSIS  PLAN 

The  plan  used  to  generate  and  analyze  the  data  that  answered  the  first  research 
question  was  produced  by  Richardson  for  his  thesis.  Recall  that  the  Measure  of 
Effectiveness  (MOE),  referred  to  earlier  as  the  MOV,  he  selected  was  the  difference  in 
total  TTTs  between  the  X  and  O  sides  in  the  assigned  mission  areas.  The  mission  areas 
selected  for  both  teams  were  VL  and  VC.  Conflict  resolution  for  same  turn  conflicts  was 
random,  P(W)  =  0.5,  for  both  initial  and  subsequent  conflicts.  The  player  strategies 
assigned  as  diescribed  in  chapter  two  were  OC  for  both  left  side  players  and  DC  for  both 
right  side  players.  Recall  also  that  Area,  Communication  and  Tactical  delays  between 
zero  and  nine  turns  were  applied  to  the  0  side  only.  Thirty  replications  were  run  for 
each  configuration  totaling  10*3*30  =  900  trials. 

After  question  one  was  answered,  it  was  determined  that  Area  and  Communication 
Delays  at  any  level  and  delay  levels  above  one  for  Tactical  Delay  were  not  statistically 
significant,  Tli'^reiOre  they  were  not  considered  while  generating  data  to  answer  the  final 
two  questions.  Unlike  the  plan  that  was  used  to  answer  the  first  research  question,  same 
turn  conflicts  were  no  longer  resolved  randomly  with  P(W)  =0.5.  A  change  in  the  T4 
software  enables  the  user  to  select  the  P(W)  for  the  O  team  on  each  side  of  the  game 
board.  Four  inputs  are  available.  The  user  can  assign  a  P(W)  for  the  initial  same  turn 
conflict  for  the  left  side  O  player  and  a  P(W)  for  subsequent  same  turn  conflicts  for  the 
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same  player.  The  same  options  are  available  for  the  right  side  O  player.  The  T4 
software  assigns  the  complementary  P(W)  to  the  X  player  automatically.  In  this  analysis, 
P(W)  was  set  the  same  for  each  player  on  a  team  for  both  initial  and  subsequent  same 
turn  conflicts.  P(W)  ranged  from  1 .0  to  0.0  in  increments  of  0. 1  for  a  total  of  eleven 
categories.  The  T4  simulation  was  run  at  these  assigned  P(W)s  for  each  delay  level  (0, 
1,  2  and  3)  for  120  replications  per  configuration  resulting  in  a  total  of  11*4’^120  = 
5280  trials. 

B.  METHODOLOGY 

This  section  will  discuss  the  statistical  methods  used  to  analyze  the  generated  data. 
Not  all  the  methods  below  were  used  to  answer  each  question.  The  methods  used  will 
be  referred  to  in  the  results. 

1.  Distribution 

The  techniques  used  to  analyze  the  data  required  an  assumption  if  they  were 
to  be  valid;  The  population  from  which  the  sample  is  drawn  must  closely  resemble  a 
normal  distribution.  The  first  requirement  then  was  to  prove  that  the  data  was  at  least 
approximately  normal. 

According  to  the  Central  Limit  Theorem  (C.L.T.),  given  a  random  sample 
from  a  distribution  with  mean  n  and  variance  cr,  if  the  sample  size  is  sufficiently  large, 
the  sample  mean  has  approximately  a  normal  distribution.  A  rule  of  thumb  for 
sufficiently  large  is  a  sample  size  greater  than  30. 
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Rather  than  just  assume  normality  based  on  the  C.L.T.,  two  simple  graphical 
techniques  were  used  to  support  this  assumption.  The  first  technique  was  the  histogram. 
The  histogram  provides  a  pictorial  representation  of  the  frequency  distribution.  If  the 
data  is  normal,  the  tops  of  the  columns  generated  by  the  histO£,iam  can  be  connected  with 
a  smooth  line  and  the  resulting  curve  will  look  similar  to  the  standard  normal  curve.  The 
second  technique  was  the  normal  probability  plot,  which  was  generated  to  supplement 
the  histogram  in  checking  for  normality.  A  normal  probability  plot  that  is  close  to  a 
straight  line  suggests  that  the  assumption  of  a  normal  distribution  is  plausible.  When 
these  techniques  show  that  the  data  is  normal,  the  following  tools  can  be  used  for  the 
data  analysis. 

2.  Single-Factor  Analysis  of  Variance 

Analysis  of  variance,  ANOVA,  is  a  statistical  tool  that  can  be  used  to  study 
the  relation  between  a  dependent  variable,  average  MOV  in  this  case,  and  one  or  more 
independent  variables.  The  independent  variables  are  called  factors  and  are  represented 
by  different  types  of  delay  and  P(W)  in  this  thesis.  Single-Factor  ANOVA  investigates 
one  factor  at  a  time  while  the  other  factor  is  held  constant  to  see  if  the  factor  being 
investigated  has  any  effect  on  the  dependent  variable.  The  question  of  central  interest 
using  ANOVA  is  whether  there  are  differences  in  the  sample  means  (ji,  ’s)  associated  with 
different  combinations  of  delay  and  P(W).  The  sample  mean  being  referred  to  is  the 
mean  MOV.  ANOVA  determines  whether  the  average  MOV  for  a  particular  level  of 
delay,  a,  is  statistically  different  than  the  average  MOV  for  some  other  delay  level,  a^, 
while  holding  P(W)  constant.  ANOVA  will  investigate  all  possible  pairs  of  delay,  ao  = 
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El,  ao  =  a2,  ...  etc.,  and  determine  if  the  average  MOV  between  different  pairs  of  a  are 
the  same  or  different.  To  answer  this  question  the  following  null  and  alternative 
hypotheses  are  tested. 

1.  H„:  /t,  =  M2  =  •  •  •  = 

2.  H,:  at  least  two  of  the  Mi’s  are  different. 

If  is  accepted,  it  can  be  concluded  that  delay  has  no  effect  on  average 
MOV.  If  is  accepted,  then  average  MOV  differs  among  different  levels  of  delay. 

The  method  used  for  choosing  between  and  is  evaluation  of  the  F 
statistic.  In  general,  large  values  of  F  support  while  values  near  one  support  Hg. 
Both  Minitab  and  Microsoft  Excel  generate  the  F  test  statistic.  To  determine  precisely 
which  alternative  should  be  chosen,  the  F  test  statistic  is  compared  with  the  F  critical 
value.  The  F  critical  value  is  generated  by  Excel  but  Minitab  users  must  look  it  up  in 
tables  of  F  critical  values  which  are  available  in  any  statistics  text.  If  the  F  test  statistic 
is  less  than  the  F  critical  value,  Hg  is  accepted.  Conversely,  if  the  test  statistic  is  greater 
than  the  critical  value,  Hg  is  accepted.  It  should  be  pointed  out  that  the  F  critical  value 
depends  on  the  assigned  value  of  the  significance  level  a.  As  the  value  of  a  decreases, 
the  F  critical  value  increases.  For  example,  if  Hg  is  just  barely  rejected  at  a  =  0.05,  it 
is  possible  that  at  a  =  0.01  the  F  critical  value  would  become  larger  than  the  F  test 
statistic  and  Hg  could  be  accepted. 
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3.  Two-Factor  ANOVA 


Since  single-factor  ANOVA  investigates  the  effect  of  one  factor  at  a  time 
while  holding  other  factors  constant,  it  has  some  limitations.  To  investigate  the  effects 
of  both  factors,  delay  and  P(W)  simultaneously,  a  stronger  analysis  tool  is  required. 
Two-factor  ANOVA  provides  this  capability  along  with  some  important  advantages  over 
single- factor  ANOVA. 

Two-factor  ANOVA  is  more  efficient  than  its  single-factor  counterpart. 
Single-factor  ANOVA  provides  no  information  about  the  factor  being  held  constant.  As 
a  result,  numerous  single-factor  runs  had  to  be  run  in  order  to  investigate  all  possible 
P(W)  and  delay  combinations.  Two-factor  ANOVA  allows  all  data  to  be  stored  in  a 
matrix  and  analyzed  in  a  single  run. 

The  single-factor  study  also  provides  less  information  than  the  two-factor 

study.  Although  numerous  single-factor  runs  were  generated,  no  information  about  any 

special  joint  effects  between  delay  and  P(W)  were  provided.  These  joint  effects  are 

called  interactions  and  indicate  that  the  effects  of  each  factor  should  not  be  discussed 

separately.  When  interactions  exist,  it  must  be  determined  if  they  are  important. 

The  determination  of  whether  interactions  are  important  or  unimportant  is 
admittedly  sometimes  difficult.  This  decision  is  not  a  statistical  decision  and 
should  be  made  by  the  subject  area  specialist  (researcher).  The  advantage  of 
unimportant  (or  no)  interactions,  namely,  that  one  is  then  able  to  analyze  the  factor 
effects  separately,  is  especially  great  when  the  study  contains  more  than  two 
factors.  [Ref.  5:p.  687] 

Like  single-factor  ANOVA,  two-factor  ANOVA  also  has  null  and  alternate 
hypotheses.  However,  they  are  not  as  simple  as  single-factor  hypotheses  and  a 
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discussion  of  the  parameters  used  in  them  is  necessary  before  they  are  stated.  In  two- 
factor  ANOVA,  values  known  as  interaction  parameters  and  main  effects  are  what  is 
compared.  Let: 

1.  M  =  1/U 

2.  =  1/J  and 

3.  /i.j  =  l/I  Ei  /ijj. 

Where  I  and  J  represent  the  different  levels  of  factor  A  and  B.  Factor  A  is 
Tactical  Delay  and  has  I  =4  levels,  while  factor  B  is  P(W)  and  has  J  =  11  levels.  The 
data  matrix  then  is  I  columns  by  J  rows  with  each  cell  containing  120  MOVs,  the  number 
of  replications  for  each  Tactical  Delay-P(W)  combination.  Thus  n  is  the  true  grand 
mean,  /tj.  is  the  expected  response  averaged  over  all  levels  of  one  factor  while  the  other 
factor  is  held  constant,  and  the  same  is  true  for  ^  Therefore,  /ti.  is  the  mean  of  a 
column  of  data  representing  a  constant  time  delay  over  all  values  of  P(W). 

Now  define: 

1.  a,  =  fii  -  n  =  the  effect  of  factor  A  at  level  /, 

2.  jSj  =  /*.j  -  /*  =  the  effect  of  factor  B  at  level  j,  and 

3-  7ii  =  -  0*  +  «!+  /3j)=  the  interaction  parameter  at  level  ij. 

There  are  now  three  sets  of  hypotheses  that  will  be  considered: 

1.  7®  =  0  for  all  ij  versus  at  least  one  7^  ^  0 

2.  Hoa:  =~.  =  a4  =  0  versus  H^:  at  least  one  aj  0 
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3.  Hog:  (5,  =  —  =  =0  versus  H,*:  at  least  one  ^  0 


The  interaction  hypothesis,  Hoab.  ‘S  tested  first  to  determine  if  interactions 
exist.  This  hypothesis  addresses  Question  4.  If  H^ab,  is  rejected  and  the  interactions  are 
determined  to  be  important,  straightforward  interpretation  of  the  results  is  not  possible. 
However,  if  the  interactions  exist  and  are  deemed  unimportant,  the  two  factors  can  be 
analyzed  separately. 

4.  Tukey’s  Procedure 

If  Ho  is  rejected  using  single-factor  ANOVA,  it  is  important  to  know  which 
n/s  are  significantly  different  from  one  another.  Determining  which  n/s  are  different 
will  show  how  the  different  factor  levels  effect  mission  outcome.  To  conduct  this  further 
analysis,  a  multiple  comparison  procedure  will  be  used.  There  are  a  number  of  these 
procedures  in  statistics  literature.  The  one  that  will  be  used  in  this  thesis  is  called 
Tukey’s  Procedure.  This  procedure  produces  a  collection  of  simultaneous  confidence 
intervals  about  the  true  values  of  all  pairwise  differences  between  every  sample  mean. 
If  an  interval  does  not  contain  zero  it  can  be  concluded  that  the  two  means  being 
compared  differ  significantly  at  level  a. 

5.  Simple  Linear  Regression  and  Correlation 

The  objective  of  regression  analysis  is  to  exploit  the  relationship  between  two 
(or  more)  variables  so  that  we  can  gain  information  about  one  of  them  through  knowing 
values  of  the  other(s)  [Ref.  6:p.  4S3].  Minitab  uses  the  least  squares  estimate  method 


28 


to  generate  a  fitted  regression  line.  The  slope  of  the  regression  line  provides  insight  into 
the  relationship  of  the  independent  variable  to  the  dependent  variable,  the  average  MOV. 
If  the  slope  is  zero,  the  line  is  horizontal  and  the  average  MOV  is  constant  and  does  not 
depend  on  the  independent  variable,  delay. 

The  sample  correlation  coefficient  measures  how  strong  the  association  is 
between  two  variables  in  a  sample.  It  is  always  between  -1  and  +1.  A  positive 
correlation  occurs  when  the  MOV  tends  to  increase  as  delay  increases.  A  negative 
correlation  implies  the  opposite.  If  there  is  almost  no  association  between  the  two 
variables,  then  the  sample  correlation  coefficient  will  be  near  zero. 

The  methods  above  were  used  to  make  the  final  conclusions  about  whether 
to  accept  or  reject  the  null  hypotheses.  However,  it  is  always  wise  to  graph  data  prior 
to  applying  more  sophisticated  analysis  techniques.  Graphing  data  provides  a  preliminary 
idea  of  what  to  expect  from  the  analysis.  Graphs,  when  compared  to  the  results  of 
statistical  methods,  provide  a  visual  description  of  what  the  results  mean.  Finally, 
graphical  representation  of  data  provides  confirmation  of  results  that  may  not  be 
understood  otherwise  and  should  always  be  produced.  In  the  following  section,  graphs 
will  be  used  in  support  of  results  wherever  applicable. 

C.  RESULTS 

1.  Time  Delay 

Research  question  one  asked  if  there  was  a  value  of  delay  above  which 
mission  outcome  for  the  O  Team  ceased  to  decline.  It  was  expected  that  at  an  increased 
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number  of  simulation  replications,  the  data  would  smooth,  clearly  indicating  the  delay 
level  above  which  mission  accomplishment  no  longer  declined.  The  results  of  single 
factor  ANOVA  will  either  prove  or  disprove  this  expectation,  but  before  considering 
these  results  a  look  at  the  data  graphically  will  provide  a  gut  feeling  for  what  to  expect 
from  the  analysis. 

a.  Graphical  Analysis 

Figure  1  is  a  graph  of  the  average  MOV  for  each  of  the  three  types  of 
delay  plotted  at  a  constant  P(W)  =  0.5.  This  graph  shows  by  how  much  on  average  the 
X  side  beats  the  O  side  as  intelligence  delay  applied  to  team  O  increases  for  each  of  the 
three  types  of  delay.  The  graph  indicates  a  clear  increase  in  average  MOV  between  delay 
levels  zero  and  one  for  Tactical  Delay,  indicating  an  increase  in  mission  outcome  for  the 
team.  It  is  difficult  to  determine  graphically  though,  if  there  is  any  significant  change 
in  the  average  MOV  among  Tactical  Delays  beyond  one.  The  graphs  of  Area  and 
Communication  Delay  do  not  indicate  any  significant  relationships  are  present. 

b.  Normality  Checks 

To  answer  the  first  research  question,  single-factor  ANOVA  was  used. 
Before  conducting  ANOVA,  the  data  was  checked  for  normality.  The  histograms 
generated  for  question  one  data  produced  columns  similar  to  the  standard  normal  curve 
and  the  normal  probability  plots  generated  fairly  straight  lines.  An  example  of  each  is 
in  Appendix  A.  Given  the  large  sample  size  and  the  positive  results  of  the  graphical 
techniques,  the  assumption  that  the  data  was  normal  appears  valid. 
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Figure  1.  Average  MOV  vs.  Time  Delay 
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c.  Single-Factor  ANOVA 


Appendix  B  has  the  results  of  the  single-factor  ANOVA  conducted  on 
each  set  of  data.  In  the  appendix  Groups  equate  to  time  delays  applied  to  team  O. 
Comparison  of  the  F  test  statistic  against  the  F  critical  value  reveals  the  null  hypotheses 
for  Area  and  Communication  delays  were  accepted  indicating  no  statistical  difference 
between  average  MOVs  at  any  level  of  delay.  Comparison  of  the  F  values  for  Tactical 
Delay  indicates  rejection  of  Ho,  thus  requiring  turther  analysis  of  the  data  to  determine 
at  which  level(s)  of  Tactical  Delay  average  MOVs  differed. 

Before  discussing  any  further  analysis,  a  more  thorough  discussion  of 
some  of  the  ANOVA  table  is  presented. 

The  Summary  section  of  the  output  consists  of  five  columns  and  ten  rows. 
The  first  column.  Groups,  represents  each  level  of  delay.  Tactical  Delays  zero  through 
nine  and  the  Count  column  indicates  the  number  of  replications  of  each  level.  The  total 
number  of  X  team  TTTs  minus  O  team  TTTs  per  replication,  summed  over  all 
replications  is  in  the  Sum  column  while  that  same  number  divided  by  the  count  is  stored 
in  the  Average  column.  This  column  is  the  average  MOV.  Finally,  the  Variance  column 
contains  the  sample  variance,  s^,  about  the  average.  Along  with  the  assumption  that  the 
data  is  normal,  an  additional  underlying  assumption  made  in  order  to  use  ANOVA  is  that 
the  different  levels  of  delay  have  the  same  population  variance,  a^.  Since  is  usually 
not  known,  s^  is  used.  The  results  show  that  the  differences  between  them  are  negligible. 
However,  if  the  assumptions  about  normality  or  variance  seem  doubtful,  a  knowledgeable 
statistician  should  be  consulted. 
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The  section  labeled  ANOVA  contains  the  values  that  determine  which 


hypothesis  is  accepted.  The  first  value  in  this  column  represents  the  mean  square  for 
treatments,  MSTr.  As  the  average  MOVs  become  more  discrepant,  this  value  increases 
indicating  H,  is  probably  true.  If  (P'  were  available  it  could  be  compared  directly  to 
MSTr.  If  MSTr  was  considerably  larger  than  <P  ,  would  be  rejected.  The  second 
value  in  this  column  represents  the  mean  square  error,  MSE.  This  value  is  an  unbiased 
estimator  of  <P  and  is  the  weighted  average  of  the  sample  variances  in  the  above  section. 
The  F  test  statistic  is  the  result  of  dividing  MSTr  by  MSE. 

The  next  section  in  the  ANOVA  table  to  be  discussed  is  the  column 
labeled  SS,  sum  of  squares.  The  first  value  in  this  column  is  the  treatment  sum  of 
squares,  SSTr,  the  second  value  is  the  error  sum  of  squares,  SSE,  and  the  third  value  is 
the  sum  of  the  first  two  and  is  appropriately  named  the  total  sum  of  squares,  SST.  The 
SST  is  a  measure  of  the  total  variation  in  the  data.  The  SSE  measures  the  variation 
within  the  different  levels  of  delay  and  is  the  unexplained  part  of  the  SST.  SSTr 
measures  the  variation  between  the  different  delay  levels,  and  is  the  explained  part  of  the 
SST.  Like  the  MS  values,  the  ratio  of  the  SS  values,  if  each  is  divided  by  the  correct 
degrees  of  freedom,  df,  can  determine  the  F  test  statistic. 

d.  Tukey*s  Method 

Tukey’s  method  was  used  to  generate  a  collection  of  confidence  intervals 
about  the  true  value  of  all  pairwise  differences  between  average  MOVs  for  Tactical 
Delay,  see  Appendix  C.  It  is  up  to  the  researcher  to  investigate  the  intervals  and  identify 
those  that  do  not  contain  zero.  In  the  appendix,  row  and  column  labels  represent  delay 
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levels.  The  results  show  that  Tactical  Delay  of  zero,  when  compared  with  every  other 
level  of  delay,  was  significantly  different  than  all  other  levels.  For  example,  the  interval 
comparing  delay  levels  zero  and  one  ranges  from  -2.839  to  -0.095  and  does  not  include 
zero.  Tukey’s  analysis  indicates  there  is  no  significant  difference  in  average  MOV  past 
a  delay  of  one  turn.  Tukey’s  method  by  itself  seemed  to  answer  research  question  one. 
However,  the  graph  of  average  MOV  vs  Time  Delay,  Figure  1,  appears  to  indicate  large 
differences  between  average  MOVs  beyond  one  for  Tactical  Delay.  A  straighter,  more 
horizontal  curve  was  expected  if  there  were  no  significant  differences  between  the 
average  MOVs  at  delay  levels  one  through  nine.  To  confirm  Tukey’s  results  and  to 
discount  the  visual  analysis,  a  fitted  regression  line  was  plotted  over  the  delay  levels  of 
interest  to  analyze  the  association  between  MOVs  at  these  delay  levels.  The  correlation 
coefficient,  r,  was  also  computed. 

e.  Corroboration  of  Tukey*s  Method 

Simple  linear  regression  was  used  to  generate  a  regression  equation,  see 
Appendix  D,  to  analyze  the  average  MOV  beyond  Tactical  Delay  of  one.  The  equation 
that  is  generated  represents  a  line  that  is  very  nearly  horizontal,  slope  =  0.065, 
indicating  the  MOV  does  in  fact  remain  fairly  constant  after  a  delay  level  of  one.  Along 
with  the  results  of  linear  regression,  a  correlation  coefficient  was  generated.  The 
coefficient  is  close  to  zero,  r  =  0.04,  indicating  there  is  almost  no  association  between 
Tactical  Delay  and  average  MOV.  This  corroborates  the  Tukey  results,  i.e.,  the  only 
difference  between  average  MOV  is  the  difference  that  exists  between  Tactical  Delay 
levels  zero  and  one. 
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2.  Changing  P(W) 


The  response  to  question  one,  showed  that  only  Tactical  Delay  effected 
average  MOV.  Further,  using  Tukey’s  method  it  was  further  determined  that  significant 
differences  in  average  MOV  only  occurred  between  Tactical  Delay  levels  zero  and  one. 
It  was  expected  that  the  results  from  Question  1  analysis  would  allow  elimination  of 
insignificant  delay  levels.  Analysis  showed  that  significant  changes  ceased  to  exist 
beyond  a  delay  of  one.  The  remainder  of  the  research  questions  introduce  an  additional 
factor,  P(W).  There  was  uncertainty  regarding  whether  including  P(W)  would  change 
the  results  beyond  a  delay  of  one.  To  protect  against  this  outcome  Tactical  Delay  levels 
from  zero  to  three  were  used  for  the  remainder  of  the  analysis.  The  second  research 
question  investigates  the  relationship  between  MOV  and  P(W)  as  P(W)  varies  from  1.0 
to  0.0  in  increments  of  0.1  while  the  levels  of  Tactical  Delay  vary  from  zero  to  three. 

a.  Normality  Checks 

This  configuration  was  also  analyzed  using  single-factor  ANOVA.  The 
data  was  checked  for  normality  and  the  results  of  histograms  and  normal  probability  plots 
confirmed  it.  Single-factor  ANOVA  was  run  at  each  level  of  Tactical  Delay  to  determine 
the  effect  of  changing  P(W)s  on  average  MOV  at  constant  levels  of  Tactical  Delay. 

b.  Graphical  Analysis 

The  graphical  analysis  is  based  on  Figure  2  which  shows  that  a  steady 
increase  in  MOV  for  the  X  team  occurs  as  the  P(W)  for  the  O  team  decreases.  It  also 
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Flgiire  2.  Avarage  MOV  vs.  P(W) 
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P(Winning  Same  Turn  Conflict  O  Team) 


appears  that  increasing  delay  level  for  team  O  results  in  a  higher  MOV  for  team  X, 
although  the  differences  appear  slight  and  are  probably  insignificant. 

c.  Single-Factor  ANOVA 

Based  on  ANOVA  the  null  hypothesis  was  rejected  all  four  times 
indicating  that  P(W)  significantly  effects  average  MOV  at  all  four  levels  of  delay.  To 
determine  which  different  levels  of  P(W)  effect  the  average  MOV,  Tukey’s  method  was 
used  once  again. 

d.  Tukey*s  Method 

Tukey’s  method  generated  confidence  intervals  revealing  how  pairwise 
levels  of  P(W)  effected  average  MOVs.  The  results  are  in  Appendix  F.  These  results 
indicate  that  the  majority  of  the  pairwise  comparisons  have  a  significantly  different  effect 
on  average  MOV.  At  Tactical  Delay  zero  for  example,  just  three  pairs  of  P(W)  have 
the  same  effect  on  average  MOV  at  a  =  0.05.  The  interval  comparing  P(W)  =  0.6  and 
P(W)  =  0.5  contains  zero,  indicating  the  average  MOV  does  not  differ  significantly 
between  those  two  P(W)s.  It  can  be  stated  then  that  at  Tactical  Delay  0,  no  significant 
increase  in  average  MOV  would  exist  between  P(W)  =  0.5  and  P(W)  =  0.6.  The  same 
can  be  said  for  all  intervals  that  do  not  contain  zero. 

e.  Linear  Regression 

Simple  linear  regression  was  used  once  again  to  determine  regression 
equations  for  the  data  at  each  level  of  delay.  See  Figure  3.  The  slopes  of  these 
equations  indicate  by  how  much  an  increase  of  0. 1  in  P(W)  contributes  to  an  increase  in 
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the  average  MOV  for  the  X  team.  Based  on  these  equations,  it  appears  that  a  0.1 
increase  in  P(W)  increases  the  average  MOV  by  approximately  2.0  TTTs. 

The  R-sq  (r-squared)  value,  which  is  the  correlation  coefficient  squared  is 
shown  in  Appendix  G  contains  along  with  the  regression  equations.  For  example,  at 
Tactical  Delay  0,  R-sq  =  75.2%  indicating  the  regression  equation  for  that  delay  explains 
75.2%  of  the  variation  in  the  average  MOV.  The  average  R-sq  value  for  all  four  delay 
levels  is  68.35%.  From  these  results,  it  appears  P(W)  positively  influences  the  average 
MOV.  Thus,  both  of  the  second  research  questions  and  the  third  as  well  have  been 
answered. 

3.  Interactions 

Finally,  the  fourth  research  question  asks  if  an  interaction  exists  between 
Tactical  Delay  and  P(W).  To  determine  if  interactions  exist,  two-factor  ANOVA  was 
used.  The  same  data  used  in  Question  2  was  used  again  so  the  requirement  that  the  data 
be  normal  has  already  been  confirmed.  The  data  were  manipulated  into  a  single  I  by  J 
matrix  to  enable  the  software  to  analyze  it.  ANOVA  was  then  run  on  the  data  and  the 
results  are  in  Appendix  H.  The  area  of  interest  in  the  ANOVA  results  is  the  row  labeled 
interaction.  Notice  the  F  test  statistic  there  is  greater  than  the  F  critical  value,  implying 
that  Hoab  is  rejected  and  interactions  between  delay  and  P(W)  do  exist.  In  following  the 
methodology  used  in  the  previous  two  questions,  the  data  was  graphed  and  then 
compared  with  the  results  provided  by  ANOVA. 
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a.  Graphical  Results 

Figure  2  is  also  a  graph  of  the  interactions.  If  no  interactions  existed, 
the  four  curves  would  be  parallel  and  wouldn’t  intersect.  Since  they  do  intersect  and  the 
results  of  ANOVA  show  that  interactions  exist,  it  is  up  to  the  researcher  to  determine  if 
the  interactions  are  important.  If  they  are  not,  the  factors  can  be  analyzed  separately  and 
the  results  of  Question  2  are  all  that  need  to  be  considered  when  analyzing  this  set  of 
data. 

To  determine  if  interactions  are  important  it  is  necessary  to  see  how 
strong  the  interactions  are.  Figure  2  shows  that  all  four  curves  plot  similarly  and 
generally  have  about  the  same  slope.  The  graph  reveals  overall  a  delay  level  of  three 
results  in  a  larger  average  MOV  than  a  delay  of  two.  The  same  is  true  when  two  is 
compared  to  one  and  one  to  zero.  Closer  investigation  of  Figure  2  indicates  that  eight 
times  out  of  eleven,  the  average  MOV  is  greatest  for  delay  level  three  and  decreases  in 
order  down  to  level  zero. 

b.  Regression  Analysis 

To  investigate  the  graph  further,  the  regression  equations  generated 
earlier  were  once  again  investigated,  see  Figure  3  or  Appendix  G.  Comparison  of  the 
slopes  will  indicate  how  away  from  parallel  the  four  curves  are.  The  slopes  and  y- 
intercepts  for  all  four  curves  are  very  similar  indicating  the  best  fit  curves  are  very  close 
to  parallel. 
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c.  Additional  Analysis 

Recall  the  R-sq  value  shows  the  amount  of  variation  between  the  data  sets 
that  can  be  explained  by  least  squares  regression.  The  results  here  are  similar  ranging 
from  a  55%  to  =  75%.  Additionally,  the  F  values  were  compared  to  see  if  Hoab  was 
rejected  hy  a  large  amount.  The  F  test  statistic  is  relatively  close  to  the  F  critical  value 
indicating  HaA.B  was  not  strongly  rejected. 

d.  Conclusions 

Based  on  Figure  2,  the  regression  equations,  the  similarity  between 
correlation  coefficients  and  the  small  margin  between  the  F  test  statistic  and  the  F  critical 
value,  it  is  the  opinion  of  the  author  that,  although  interactions  exist  between  Tactical 
Delay  and  P(W),  they  are  unimportant  and  do  not  effect  the  average  MOV. 

e.  Multiple  Regression 

Finally,  a  multiple  linear  regression  was  run  on  the  data  to  analyze  the 
combined  influence  of  P(W)  and  Tactical  Delay  on  average  MOV  over  the  entire  I  by 
J  matrix.  The  regression  equation,  see  the  fourth  equation  in  Figure  3  above,  indicates 
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that  P(W)  has  more  than  three  times  the  effect  on  average  MOV  than  Tactical  Delay. 
This  further  corroborates  the  the  graphical  analysis  that  shows  the  almost  complete 
dominance  of  P(W)  over  delay. 

This  chapter  discussed  the  methodology  used  to  analyze  the  data  and  the 
results  of  the  analysis.  The  final  chapter  of  this  thesis  will  include  the  conclusions,  a 
summary  of  the  experiment  and  some  recommendations  for  further  study  in  this  area. 


IV.  CONCLUSIONS 


The  purpose  of  this  thesis  was  to  design,  conduct,  analyze  and  report  the  results 
of  a  C3  experiment.  The  T4  Simulation  generated  data  for  analysis  so  the  research 
questions  in  the  first  cluqiter  could  be  answered.  The  remainder  of  this  chapter 
summarizes  the  thesis  results  presented  earlier. 

A.  RESEARCH  QUESTION  RESULTS 

The  data  generated  and  analysis  conducted  was  sufficient  to  answer  the  stated 
research  questions.  The  last  chapter  discussed  the  data  analysis  and  provided  conclusions 
to  the  research  questions  posed  in  the  first  chapter.  Below  the  questions  are  answered 
directly,  a  brief  summary  of  the  experiment  is  provided  and  the  thesis  closes  with 
recommendations  for  further  studies  in  this  area. 

1.  Delay  Levels 

It  was  determined  that  Tactical  Delay  of  zero  was  significantly  different  than 
delays  one  through  nine.  Analysis  of  delays  one  through  nine  indicated  no  significant 
change  in  average  MOV  among  those  delays.  Area  and  Communication  Delays  showed 
no  significant  difference  in  average  MOV  at  any  level.  Therefore,  based  on  the 
configuration  of  this  experiment,  it  can  be  concluded  that: 

1.  Only  Tactical  Delay  significantly  effects  average  MOV,  and 

2.  There  is  a  level  of  delay  beyond  which  average  MOV  ceases  to  decline  and  it 
has  been  determined  to  be  Tactical  Delay  =  1. 
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2.  P(W) 


This  section  will  answer  research  questions  2a,  2b  and  3.  In  all  cases, 
changing  the  level  of  P(W)  provides  a  distinct  increase  in  average  MOV  for  the  team 
assigned  the  greater  value  of  P(W).  The  average  MOV  increases  linearly  with  an 
increase  in  P(W)  and  at  no  level  does  a  further  increase  in  P(W)  not  result  in  an  increase 
in  average  MOV.  It  was  determined  that  an  increase  of  0.1  in  P(W)  results  in  an 
increase  of  approximately  two  TTTs.  These  results  remained  approximately  the  same 
regardless  of  the  level  of  Tactical  Delay. 

3.  Delay  and  P(W)  Interactions 

Finally,  it  was  determined  that  an  interaction  existed  between  Tactical  Delay 
and  P(W).  The  interaction  was  determined  to  be  unimportant  and  the  two  factors  were 
analyzed  separately.  Separate  analysis  was  accomplished  using  single-factor  ANOVA 
previously  and  no  further  analysis  was  required. 

B.  SUMMARY 

This  experiment  enabled  the  researcher  to  use  several  statistical  tools  to  analyze 
data  generated  by  the  T4  Simulation.  The  simulation,  acting  as  a  surrogate  for  a  C3 
system,  provides  data  which  allows  the  researcher  to  determine  an  optimal  combination 
of  the  assigned  variables,  i.e.,  an  optimal  system  alignment.  In  this  specific 
configuration  the  X  team  player  assigned  P(W)  =  1.0  playing  against  an  O  team  assigned 
Tactical  Delay  one  or  greater  will  always  be  more  successful  than  any  other 
configuration. 
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This  type  of  experiment  lends  itself  well  to  systems  evaluation.  For  example,  if 
a  C3  system  were  being  analyzed  to  try  and  determine  the  optimal  setting  of  system 
parameters,  an  analysis  similar  to  this  one  could  be  conducted  and  an  optimal  systems 
alignment  could  be  determined  statistically. 

C.  RECOMMENDATIONS 

The  T4  Simulation  is  an  excellent  tool  for  generating  C3  related  data  to  determine 
an  optimal  T4  configuration.  The  number  of  configurations  are  numerous  and  the 
spreadsheet  generated  is  full  of  data  to  be  analyzed.  The  only  recommendations  this 
author  can  suggest  is  to  take  advantage  of  the  different  configurations  available  in  the  T4 
simulation. 

There  are  several  options  for  a  continuation  to  this  thesis.  Chapter  two  points  out 
several  parameters  that  were  not  investigated.  Another  researcher  may  attempt  to 
replicate  the  above  results  using  a  completely  different  configuration.  After  the  systems 
evaluation  course,  C3  students  will  be  very  familiar  with  this  simulation  and  may  want 
to  propose  a  configuration  that  interests  them,  predict  what  they  think  will  happen,  and 
test  their  predictions  using  some  or  all  of  the  tools  presented  in  this  thesis.  There  are 
vast  opportunities  for  the  C3  student  who  enjoys  this  type  of  analysis  to  pursue  this  area 
for  thesis  study. 
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APPENDIX  A:  CHECKING  FOR  NORMALITY 
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Histogram  of  Communication  Delay  at  level  0,  N  =  120 
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Histogram  of  Tactical  Delay  at  level  0,  N  »  120 
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APPENDIX  B;  SINGLE-FACTOR  ANOVA 


Anova:  Single-Factor 

Area  Delay 

Sumnuuy 

Groups 

Count 

Sum 

Average  Variance 

0 

120 

25 

0.208 

8.318 

1 

120 

8 

0.067 

6.802 

2 

120 

37 

0.308 

8.148 

3 

120 

59 

0.492 

8.336 

4 

120 

83 

0.692 

7.022 

5 

120 

64 

0.533 

5.965 

6 

120 

61 

0.508 

5.798 

7 

120 

82 

0.683 

6.336 

8 

120 

47 

0.392 

6.660 

9 

120 

101 

0.842 

8.118 

ANOVA 

Source  of  Variation 

SS  df  MS  F  P-val  F  crit 

Between  Groups  60.3  9  6.7  0.94  0.492  1.89 

Within  Groups  8S08.8  IISK)  7.2 

Total  8569.1  1199 


Anova:Single-Factor  G^mm  Delay 


Sununary 


Groups 

Count 

Sum 

Average  Variance 
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120 

32 

0.27 

10.6 

1 

120 

-3 

-0.025 

7.991 

2 

120 

-4 

-0.033 

7.427 

3 

120 

46 

0.383 

8.205 

4 

120 

27 

0.225 

6.058 

5 

120 

9 

0.075 

7.818 

6 

120 

58 

0.483 

6.806 

7 

120 

53 

0.442 

7.912 

8 

120 

-7 

-0.058 

7.114 

9 

120 

7 

0.058 

7.770 
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ANOVA 


Source  of  Variation 

SS  df  MS  F  P-val  F  crit 

Between  Groups  45,8  9  5.1  0.655  0.75  1.89 

Within  Groups  9242.6  1190  7.77 

Total  9288.397  1199 


Anova:  Single-Factor 

Tactical  Delay 

Summary 

Groups 

Count 

Sum 

Average 

Variance 

0 

120 

-7 

-0.058 

7.971 

1 

120 

169 

1.408 

10.328 

2 

120 

201 

1.675 

9.986 

3 

120 

185 

1.542 

11.679 

4 

120 

263 

2.192 

10.711 

5 

120 

244 

2,033 

12.520 

6 

120 

253 

2.108 

10.770 

7 

120 

260 

2.167 

13.165 

8 

120 

176 

1.467 

13.327 

9 

120 

270 

2.250 

12.609 

ANOVA 


Source  of  Variation 


SS 

df 

MS 

F 

P-value  F  crit 

Between  Groups 

515.1 

9 

57.23 

5.06 

0  1.89 

Within  Groups 

13454.8 

1190 

11.3 

Total 

13969.84 

1199 
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APPENDIX  C:  TUKEY’S  RESULTS 


Tukey’s  Pairwise  Comparisons 
Family  error  rate  =  0.0500 
Individual  error  rate  =  0.00160 


Critical  value  =  4.47 
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APPENDIX  D;  LINEAR  REGRESSION 


The  regression  equation  is 

Average  MOV  =  1,55  +  0.0651  Tactical  Delay 

Predictor  Coef  Stdev  t-ratio  p 

Constant  1.5456  0.2266  6.82  0.000 

TacDel  0.06514  0.04027  1.62  0.106 

s  =  3.417  =  0.2%  R-sq(adj)  =  0.1% 
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APPENDIX  E;  SINGLE-FACTOR  ANOVA 


Changing  P(W)  at  Time  Delay  0 
Anova:  Single-Factor 

Summary 


Groups 

Count 

Sum 

Average 

Variance 

1.0 

120 

-1326 

-11.050 

13.964 

0.9 

120 

-885 

-7.375 

15.682 

0.8 

120 

-550 

•4.583 

15.455 

0.7 

120 

-334 

-2.783 

10.659 

0.6 

120 

-166 

-1.383 

9.415 

0.5 

120 

-7 

-0.058 

7.971 

0.4 

120 

217 

1.808 

10.223 

0.3 

120 

360 

3.000 

7.697 

0.2 

120 

631 

5.258 

11.991 

0.1 

120 

957 

7.975 

17.319 

0.0 

120 

1322 

11.017 

13.126 

ANOVA 


P-val  F  crit 
0  1.838 


Total  67696.67  1399 


Source  of  Variation 

SS  df  MS  F 

Between  Groups  S 1809. 87  10  SI 80.987  426.89 

Within  Groups  1S886.79  1309  12.13659 
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Gianging  P(W)  at  Time  Delay  1 
Anova:  Single-Factor 


Summary 

Groups  Count  Sum  Average  Variance 


1.0 

120 

-867 

-7.225 

36.361 

0.9 

120 

-764 

-6.367 

25.125 

0.8 

120 

-475 

-3.958 

21.250 

0.7 

120 

-299 

-2.492 

15.865 

0.6 

120 

-53 

-0.442 

12.249 

0.5 

120 

169 

1.408 

10.328 

0.4 

120 

188 

1.567 

12.735 

0.3 

120 

496 

4.133 

13.696 

0.2 

120 

702 

5.850 

15.154 

0.1 

120 

1004 

8.367 

16.890 

0.0 

120 

1412 

11.767 

12.332 

ANOVA 


Source  of  Variation 

SS  df  MS  F 

Betwem  Groups  43746.7  10  4374.666  2S0.6S3 
Within  Groups  22846.1  1309  17.45311 

Total  66592.8  1319 


Changing  P(W)  at  Time  Delay  2 
Anova:  Single-Factor 


Summary 

Groups  Count  Sum  Average  Variance 


1.0 
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-1332 

-11.1 

16.242 
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120 

-709 

-5.908 

25.042 

0.8 

120 

-332 

-2.767 

22.315 

0.7 

120 

-127 

-1.058 

12.980 

0.6 

120 

33 

0.275 

14.789 

0.5 

120 

219 

1.825 

12.667 

0.4 

120 

348 

2.9 

13.183 

0.3 

120 

501 

4.175 

13.809 

0.2 

120 

861 

7.175 

17.473 

0.1 

120 

1137 

9.475 

18.336 

0.0 

120 

1467 

12.225 

9.436 

P-val  F  crit 
0  1.838 
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ANOVA 


Source  of  Varietioa 

SS  df  MS  F  P-val  F  crit 

Between  Groups  55188  10  5518.8  344.392  0  1.838 

Within  Groups  20976.4  1309  16.02475 
Total  76164.4  1319 


Qumging  P(W)  at  Time  Delay  3 
Anova:  Single-Factor 

Sununary 

Groups  Count  Sum  Average  Variance 


1.0 

120 

-849 

-7.075 

32.272 

0.9 

120 

-632 

-5.267 

30.584 

0.8 

120 

-342 

-2.85 

21.624 

0.7 

120 

-212 

-1.767 

16.668 

0.6 

120 

55 

0.458 

15.948 

0.5 

120 

257 

2.142 

10.744 

0.4 

120 

350 

2.917 

13.859 

0.3 

120 

556 

4.633 

16.217 

0.2 

120 

813 

6.775 

17.588 

0.1 

120 

1057 

8.808 

16.190 

0.0 

120 

1505 

12.542 

9.746 

ANOVA 

Source  of  Variation 

SS  df  MS  F  P-val  F  crit 
Between  Groups  43593.62  10  4359.4  238.052  0  1.838 

Within  Groups  23971.28  1309  18.3 


Total 


67564.91  1319 


APPENDIX  F:  TUKEY’S  RESULTS 


Tukey’s  painvise  comparisons  for  Tactical  Delay=  0 

Family  error  rate  =  O.OSOO 
Individual  error  rate  =  0.00132 

Critical  value  4.SS 


Intervals  for  (column  level  mean)  -  (row  level  mean) 


1.0 

0.9 

0.8 

0.7  0.6  0.5  0.4 

0.3  0.2  0.1 

0.9 

-5.122 

-2.228 

0.8 

-7.914 

-5.020 

-4.239 

-1.345 

0.7 

-9.714 

-6.820 

-6.039 

-3.145 

-3.247 

-0.353 

0.6 

-11.114 

-8.220 

-7.439 

-4.545 

-4.647 

-1.753 

-2.847 

0.047 

0.5 

-12.439 

-9.545 

-8.764 

-5.870 

-5.972 

-3.078 

-4.172 

-1.278 

-2.772 

0.122 

0.4 

-14.305 

•11.411 

-10.630 

-7.736 

-7.839 

-4.945 

-6.039 

-3.145 

-4.639 

-1.745 

-3.314 

-0.420 

0.3 

-15.497 

-12.603 

-11.822 

-8.928 

-9.030 

-6.136 

-7.230 

-4.336 

-5.830 

-2.936 

-4.505 

-1.611 

-2.639 

0.255 

0.2 

-17.755 

-14.861 

-14.080 

-11.186 

-11.289 

-8.395 

-9.489 

-6.595 

-8.089 

-5.195 

-6.764 

-3.870 

-4.897  -3.705 
-2.003  -0.811 

0.1 

-20.472 

-17.578 

-16.797 

-13.903 

-14.005 

-11.111 

-12.205 

-9.311 

-10.805 

-7.911 

-9.480 

-6.586 

-7.614  -6.422  -4.164 
-4.720  -3.528  -1.270 

0.0 

-23.514 

-20.620 

-19.839 

-16.945 

-17.047 

-14.153 

-15.247 

-12.353 

-13.847 

-10.953 

-12.522 

-9.628 

-10.655  -9.464  -7.205 
-7.761  -6.570  -4.311 

•4.489 

-1.595 
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Tukey’s  painvise  compaiisoos  for  Tactical  Delay  s  i 

Family  error  rate  O.OSOO 
Individual  error  rale  »  0.00132 

Critical  value  —  4.55 


Intervals  for  (column  level  mean)  -  (row  level  mean) 


1.0 

0.9  0.8 

0.7 

0.6 

0.5 

0.4  0.3 

0.2  0.1 

0.9 

-2.594 

0.877 

0.8 

-5.002 

-1.531 

-4.144 

-0.673 

0.7 

-6.469 

-2.998 

-5.610  -3.202 

-2.140  0.269 

0.6 

-8.519 

-5.048 

-7.660  -5.252 

-4.190  -1.781 

-3.785 

-0.315 

0.5 

-10.369 

-6.898 

-9.510  -7.102 

-6.040  -3.631 

-5.635 

-2.165 

-3.585 

-0.115 

0.4 

-10.527 

-7.056 

-9.669  -7.260 

-6.198  -3.790 

-5.794 

-2.323 

-3.744 

-0.273 

-1.894 

1.577 

0.3 

-13.094 

-9.623 

-12.235  -9.827 

-8.765  -6.356 

-8.360 

-4.890 

-6.310 

-2.840 

-4.460 

-0.990 

-4.302 

-0.831 

0.2 

-14.810 

-11.340 

-13.952  -11.544 
-10.481  -8.073 

-10.077 

-6.606 

-8.027 

-4.556 

-6.177 

-2.706 

-6.019  -3.452 
-2.548  0.019 

0.1 

-17.327 

-13.856 

-16.469  -14.060 
-12.998  -10.590 

-12.594 

-9.123 

-10.544 

-7.073 

-8.694 

-5.223 

-8.535  -5.969 
-5.065  -2.498 

-4.252 

-0.781 

0.0 

-20.727 

-17.256 

-19.869  -17.460 
-16.398  -13.990 

-15.994 

-12.523 

-13.944 

-10.473 

-12.094 

-8.623 

-11.935  -9.369 
-8.465  -5.898 

-7.652  -5.135 
-4.181  -1.665 
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Tukey’s  pairwise  compahsoas  for  Tactical  Delay  =  2 


Family  error  rate  ~  0.0500 

Individual  error  rate  =  0.00132 

Critical  value  »  4.SS 

Intervals  for  (column  level  mean)  -  (row  level  mean) 

1.0  0.9  0.8  0.7  0.6  0.5  0.4  0.3  0.2  0.1 

0.9  -6.854 

-3.529 

0.8  -9.996  -4.804 

-6.671  -1.479 

0.7  -11.704  -6.513  -3.371 

-8.379  -3.187  -0.046 

0.6  -13.038  -7.846  -4.704  -2.996 

-9.712  -4.521  -1.379  0.329 

0.5  -14.588  -9.396  -6.254  -4.546  -3.213 

-11.262  -6.071  -2.929  -1.221  0.113 

0.4  -15.663  -10.471  -7.329  -5.621  -4.288  -2.738 

-12.337  -7.146  -4.004  -2.296  -0.962  0.588 

0.3  -16.938  -11.746  -8.604  -6.896  -5.563  -4.013  -2.938 

-13.612  -8.421  -5.279  -3.571  -2.237  -0.687  0.388 

0.2  -19.938  -14.746  -11.604  -9.896  -8.563  -7.013  -5.938  -4.663 

-16.612  -11.421  -8.279  -6.571  -5.237  -3.687  -2.612  -1.337 

0.1  -22.238  -17.046  -13.904  -12.196  -10.863  -9.313  -8.238  -6.963  -3.963 

-18.912  -13.721  -10.579  -8.871  -7.537  -5.987  -4.912  -3.673  -0.637 

0.0  -24.988  -19.796  -16.654  -14.946  -13.613  -12.063  -10.988  -9.713  -6.713  -4.413 

-21.662  -16.471  -13.329  -11.621  -10.287  -8.737  -7.662  -6.387  -3.387  -1.087 
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Tukey’s  paiiwise  comparisons  for  Tactical  Delay  s  3 

Family  error  rate  =  0.0500 
Individual  error  rate  =  0.00132 

Critical  value  »  4.SS 


Intervals  for  (column  level  mean)  -  (row  level  mean) 


1.0 

0.9 

0.8 

0.7 

0.6 

0.5 

0.4 

0.3 

0.2 

0.9 

-3.586 

-0.031 

0.8 

-6.002 

-2.448 

•4.194 

-0.639 

0.7 

-7.086 

-3.531 

-5.277 

-1.723 

-2.861 

0.694 

0.6 

-9.311 

-5.756 

-7.502 

-3.948 

-5.086 

-1.531 

-4.002 

-0.448 

0.5 

-10.994 

-7.439 

-9.186 

-5.631 

-6.769 

-3.214 

-5.686 

-2.131 

-3.461 

0.094 

0.4 

-11.769 

-8.214 

-9.961 

-6.406 

-7.544 

-3.989 

-6.461 

-2.906 

■4.236 

-0.681 

-2.552 

1.002 

0.3 

-13.486 

-9.931 

-11.677 

-8.123 

-9.261 

-5.706 

-8.177 

-4.623 

-5.952 

-2.398 

-4.269 

-0.714 

-3.494 

0.061 

0.2 

-15.627 

-12.073 

-13.819 

-10.264 

-11.402 

-7.848 

-10.319 

-6.764 

-8.094 

-4.539 

-6.411 

-2.856 

-5.636 

-2.081 

-3.919 

-0.364 

0.1 

-17.661 

-14.106 

-15.852 

-12.298 

-13.436 

-9.881 

-12.352 

-8.798 

-10.127 

-6.573 

-8.444 

•4.889 

-7.669 

-4.114 

-5.952 

-2.398 

-3.811 

-0.256 

0.0 

-21.394 

-17.839 

-19.586 

-16.031 

-17.169 

-13.614 

-16.086 

-12.531 

-13.861 

-10.306 

-12.177 

-8.623 

-11.402 

-7.848 

-9.686 

-6.131 

-7.544 

-3.989 
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APPENDIX  G;  LINEAR  REGRESSION 


The  regression  equation  at  Tactical  Delay  0  is 
Average  MOV  =  -  11.6  +  1.96  P(W) 


Predictor 

Coef 

Stdev  t-ratio 

Constant 

-11.6168 

0.2107  -55.13 

C46 

1.96379 

0.03107  63.21 

8  =  3.569 

R-sq  = 

75.2%  R-sq(adj) 

The  regression  equation 
Average  MOV  =  -  9.69 

at  Tactical  Delay  i 
+  1.81  P(W) 

Predictor 

Coef 

Stdev  t-ratio 

Constant 

-9.6852 

0.2497  -38.79 

C48 

1.80523 

0.03681  49.04 

s  =  4.229 

R-sq  = 

64.6%  R-sq(adj) 

The  regression  equation  at  Tactical  Delay  2 
Average  MOV  =  -  10.5  +  2.01P(W) 

Predictor 

Coef 

Stdev  t-ratio 

Constant 

-10.4935 

0.2458  -42.69 

C50 

2.00977 

0.03624  55.46 

s  -  4.163 

R-sq  *5 

70.0%  R-sq(adj) 

The  regression  equation  at  Tactical  Delay  3  is 
Average  MOV  »  -  8.89  +  1.80P(W) 
Predictor  Cotf  Stdev  t-ratio 
Constant  -8.8903  0.2549  -34.88 

C52  1.80470  0.03758  48.02 


P 

0.000 

0.000 

=  75.2% 


P 

0.000 

0.000 

=  64.6% 


P 

0.000 

0.000 

=  70.0% 


P 

0.000 

0.000 


s  »  4.318  R-sq  »  63.6%  R-sq(adj)  =  63.6% 


APPENDIX  H:  TWO-FACTOR  ANOVA 


Anova:  Two-Factor  With  Replication 
Suounary 

DelayO  Delay  1  Delay  2  Delay  3  Total 


P(W)=1.0 


Count 

120 

120 

120 

120 

480 

Sum 

-1326 

-867 

-1332 

-849 

^374 

Average 

-11.05 

-7.23 

-11.1 

-7.075 

-36.45 

Variance 

13.96 

36.36 

16.24 

32.27 

98.84 

POV)=0.9 

Count 

120 

120 

120 

120 

480 

Sum 

-885 

-764 

-709  -632 

■2990 

Average 

-7.375 

-6.37 

-5.91 

-5.23 

-24.92 

Variance 

15.68 

25.12 

25.04 

30.58 

96.43 

P(W)«0.8 

Count 

120 

120 

120 

120 

480 

Sum 

-550 

-475 

-332 

-342 

-1699 

Average 

-4.58 

-3.96 

-2.77 

-2.85 

-14.16 

Variance 

15.46 

21.25 

22.31 

21.624  80.64 

P(W)=0.7 

Count 

120 

;20 

120 

120 

480 

Sum 

-334 

-299 

-127 

-212 

-972 

Average 

-2.78 

-2.49 

-l.C^ 

-1.77 

-8.1 

Variance 

10.66 

15.87 

12.98 

16.67  56.17 

P(W)=0.6 

Count 

120 

120 

120 

120 

480 

Sum 

-166 

-53 

33 

55 

-131 

Average 

-1.38 

-0.44 

0.275 

0.46 

-1.09 

Variance 

9.41 

12.25 

14.79 

15.95 

52.40 
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P(W)=0.5 

Count  120 

120 

120 

120 

480 

Sum  -7 

169 

21.9 

257 

638 

Average  -0.06 

1.41 

1.83 

2.14 

5.32 

Variance  7.97 

10.33 

12.67 

10.74 

41.71 

P(W)=0.4 

Count  120 

120 

120 

120 

480 

Sum  217 

188 

348 

350 

1103 

Average  1.81 

1.57 

2.9 

2.92 

9.19 

Variance  10.22 

12.74 

13.18 

13.86 

50.00 

P(W)=0.3 

Count  120 

120 

120 

120 

480 

Sum  360 

496 

501 

556 

1913 

Average  3 

4.13 

4.18 

4.63 

15.94 

Variance  7.70 

13.70 

13.81 

16.22 

51.42 

P(W)=0.2 

Count  120  120  120  120  480 

Sum  631  702  861  813  3007 

Average  5.26  5.85  7.18  6.78  25.06 
Variance  11.99  15.153  17.47  17.59  62.21 


P(W)=0.l 


Count 

120 

120 

120 

120 

480 

Sum 

957 

1004 

1137 

1057 

4155 

Average 

7.98 

8.37 

9.48 

8.81 

34.625 

Variance 

17.32 

16.89 

18.34 

16.19 

68.73 

P(W)=0.0 

Count 

120 

120 

120 

120 

480 

Sum 

1322 

1412 

1467 

1505 

5706 

Average 

11.02 

11.77 

12.23 

12.54  47.55 

Variance 

13.13 

12.33 

9.44 

9,75 

44.64 
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Total 


Count 

Sum 

Average 

Variance 

1320  1320  1320  1320 

219  1513  2066  2558 

1.83  12.61  17.22  21.32 

133.50  191.98  176.27  201.44 

ANOVA 

Source  of  Variation  SS  df  MS 

F 

P-val  F  crit 

Sample  192490.92  10  19249.09 

Column  2309.97  3  769.99 

Interaction  1847.23  30  61.57 

1204.44 

48.18 

3.85 

0  1.83 

0  2.61 
0  1.46 

Within 

83680.60  5236  15.98 

Total 

280328.72  5279 
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APPENDIX  I:  MULTIPLE  LINEAR  REGRESSION 


The  regression  equation  is 

MOV  =  -  11.6  +  1.90  P(W)  +  0.573  Delay 

Predictor  Coef  Stdev  t-ratio  p 
Constant  -11.6052  0.1745  -66.51  0.000 

P(W)  1.89587  0.01781  106.45  0.000 

Delay  0.57348  0.05037  11.38  0.000 

s  =  4.092  R-sq  =  68.5%  R-sq(adj)  =  68.5% 
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